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Scholastic Aptitude Test (SAT) v using item response theory' (IRT) in.a , 
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The results of the IRT equating method wei;e compared to those 
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chi-square-like test in pon ji^hction. with item ability regression 
plots is examined. The lack ot a jcriterion^for judging the equatings 
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r , - Backgroundjand Purpose of the Study 

Most large scale testing programs are involved in either of two^ sitviations 
that necessitate a statistical process referred to as equating. In the\^irst 

situation, multiple editions of attest have been copstructed to measure a 

\ 

. specific aptxfcude or ability at a defined level of proficiency. The Scholastic 
Aptitude Test (SAT) administered by Educational Testing Service. for the College 
Board is an exampl§ of this type of test. The various editions- of bhe SA^" 

ccmt'ain different questions but are carefully constructed to be r as similar ' 

. ' , " ' * 

in -difficulty.and contentras possible. In spite of these efforts, it is 

# * • - 7' /• % „ 

usually impossible to construct multiple forms that are of exac^y the same 

** ' * * 

dif^icultyUg^el* If some assessment of relative ability is'^fe* be'nnade ,for 

"\ *' * ' ' ' • . ; - 

students taking different editions of the same test, it is critical that a 

method ,0f equating, rendering ^comparable the scores on multip|e» editions of 

the test, be established*, When these multiple editions ate testing consent at* 

a very, similar difficulty level (such as is the cas* for the. SAT) the process 

\ • * - i mm • 

'*,is referred to in the literature and in practice as horizontal .equating. 

In the second situation', the testing program is interested in establishing 

* * • . 

a single scale that allows scores to \H4 compared for various "levels ol? an 
aptitude or ability. Typical examples are the many commercially av^ilable^ ' 
test batteries that contain tests developed for several gra4e levels. Because 



aggregate scores are 



\t* * 
, of ten compared acrpss levels (e.g* for program evaluation 



purposes) it is "critical that the scores obtained on the various leveTTs of 
the test be equated,, i.e. , placed on a common underlying scale. This type of, 
equating" is usually referred to as vertical equating; its purpose being to 

\ -;• - . • 

place on a single scale^ multiple editions of a test which are each designed 

'''J ■ . - 

to measure a different *JLevel of the same attribute.- 

^n this paper, the vertical equating situation that , will be examined is/ 4 

the equating of tfie Preliminary Scholastic AptttCle Test/National Merit Scholar 

' ' ' " . ' *V 

ship Qualifying Test (PSAT/NMSQT) tp £he-.SAT, The PSAT/NMSQT, similar to the 

SAT, is administered* by Educational Testing Service"' for College Board and the 
National M^it Scholarship Corporation. The equating of the PSAT/faMSQT repre- 
sents a fairly unique situation >in that the use of the scores requires the test 

t ' - ^ >' r I " , 

be equated both vertically and, horizontally. 

PSAT/NMSQT scores are used for two" purposes: (1) to give -examinees an' ' 
idea of what the SAT is like and what scores they might expect to obtain whei> 

they , take the SAT; and (2) as a-nieans, of screening candidates for qualifi- 

• - * *- • ** »* 

. * & * ' " - — ■! . v 

cation as National Merit Scholars. The first pufjfirse requires that sdores on 

.,*»<' 

the PSAT/NMSQT be as' comparable* as possible to score^'on the SAT (a Vfe-Eeicat -* 
equating situation). Tfoe second p.urposejrequirg^that scores on alternate 
editions of the PSAT/NMSQT be as comparable as possible (a Tiorizofctal equating 
sitrctfrironK Only .the vertical* equating of the PSAT/NHSfQT is of interest for . 
this study. The reader .is referred to Cook, Dunbar and Eignor (1981) for a 
discussion *of the horizontal* equating situation. 

'' There has been a good deal of discussion in the literature as to: j(l) the 



a mit 



appropriateness of i*tem* response theory (IRT) 4s .a mdthod to equate tests; 
(2) -the feasibility'bf using IRT equating method's in;a vertical equating situ- 
ation; and (3) the choice of IRT model (one-parameter v^stt^tbrfee-parameterX 



for test equating purposes* The following is a brief review of some of the 

relevant research. • " 

— ^ -i - . < ^ * 

- Slinde and Linn (1977^ 1973,. 1979) investigated, the problem of vertical 
equating of two forms -designed tor populations* at -different; levels of ability. 
TheJLr "results suggested that linear, . e^uipercentlle and IRT equating .employing 
the one-parameter logistic model may have limitations for the- process of vert- 
ical equating. This was especially true when ' the differences between tes? 
difficulty and between abilitf levels of equating samples were hosX pronounced. 
Their studies imply th&t an IRT approach, based on the more complex .three- 

parameter logistic model might provide more useful, results for vertical , 

• i < 

equating 'situations. - ( , 

itarco*, Petersen, and Stewart (1979) presented perhaps the mos^t compre- 
hensi^e empirical study of equating techniques yet to appear. , For designs ^ 
like the PSAT/MMSQT. design, they found problems with traditional methods 

^i^^lar to- those found in the Slinde and Linn stbdies. In parttcu^Lar, when 
tests differing in difficulty were given to non-equivalent groups and*equated 

'usitig an anchor test tiesign, traditional procedures appeared to break down. 
In spite of the presehce~-of possible, criterion bias confounding some of their 
results, the authors suggested that the three-parameter logistic model would 

V ' : - ■ .. . * ■ \; ' • - ; ' . : 

yield the most acceptable results untfer unusual or extreme design constraints.. 
^However, .Marco >t al^ Xound, as did, Slinde « an d 'Lin n, ^^J^^^^^^^ 
ilatity between groups and test forms' were ^SnrieVevaratr tJ&eii uriese factors 
wefce moderate,! traditional methods, both* linear^and equipercent|le, yielded. 

adequate eqliafclngs. . " ' 1 - 

5 \ \ J . \; ' ♦ - , / 

r A comparison of the "stability of results obtained from traditional an<£ 

IRT procedures was ma4e by Kolen (1981), who. used a cross-vali^atiori group to 




establish a crlteripn 'for the evaluation of seven IRT methods .and two tradl-r 

*» - * - » t • 

•tionai methods (linear and aqltipercentile) Kolen, working with the Iowa Tesis 

* < « ,. • 

of Educational Development, had some dif f icirl'ty evaluating the results obtained 
* • . ' ' '• : -v 

from application of the three-parameter logistic model to equate new Level I 

^tests ; <Vc^abulary and quantitative thinking tests administered to -9th and 10th 
—graders) and ney Level II 'tests* (tests of the same skills administered to 
, 11th and 12th graders) to old tests of vocabulary and quantitative thinking 
, that consisted* of one level, administered to grades 9-12. He found that % 
"Although the three-parameter estimated observed score method tended to pro- 
^iuce the most stable .cross-validation results at Level v I of the tests, the 
•result's were of.voniy moderate accuracy at/ Level II. The three-parameter 
estimated true t score equivalents method 7 tended to produce the most stable 
cross-validation results at Level II and results of moderate stability at 
\evel I." > Kolen" noted C^at these "results may be" partially explained t>y diffi- 
< cuity in estimating the psuedo-guessirig parameter of *the three-parameter model 
- and by problem? related to the' transformation of estimated true scores below the 
chance sc^re .level; he concluded that there' may be some doubt regarding the 
aj^cability of the three-jSarameter/model in all equating contexts. A- somewhat 
surprising degree of stability was found to hold, for conventional equipercentiie 
procedures, sufficient to lead Kolen to conclude that* they may be the most 
viable : in practice for designs which involve equating tests differing only 
moderately in difficulty. * * * 

More recently, Petersen, Cook, and Stocking (1981) 'investigated the rela- k 

' * • 

tive accuracy of conventional versus IRT equating methods using scale drift { 

2s the criterion f<*r evaluating tftese methods. f The' authors, used a chaihfof 

* ' ' ' ♦ t~- 

six verbal^nd :S-i^ma^hematic4l form's of 'the' SAT, each fdrnf; linked to, tike * ' 

;jpreceedln^"and' foliowUng forar- by .^anchor te?t.'* 'The design allowed them* tto 



5 - ' 



evaluate the effecp of equating a^ % st to itself via five intervening forms. 
The equating methods investigated ?^re four traditional methods (thre£ lineAr 
and one, equi'^ercefltile) and a thr^e-parameter logistic IRT method that; employed 

three procedures. for linking parameter estimates so that estim&fced true formula 

1 i * - * 

scores could be equated^ * /• 



j 



Results of the study indicate^ that ali three IRT linking methods were 



superior to the traditional method^," f.or the verbal portion of the test'. For' 

r \ 

the mathematical portion, only one of the^IRT methods produced results as 



* satisfactory as those obtained from two of .the 



three linear methods? .Equiper- 



centile equating yielded fairly unsatisfactory results foe. both the verbal and 
mathematical sections. > It should be noped ^hat the study involved tests similar 
in level of difficulty which* were given to groups- of Examinees that did not 
...differ greatly in their level of ability; a situation in whictK^ne w^uld expect 
fc traditional linear methods to work well. V J 

f s - - ■ < ' J. 

If 'anything, an in-depth look at previous research comparing various > 
equating procedures leaves the practitioner with little confirming evidence. 
On the one hand, IK* approaches, especially those using the three-parameter, 
logistic model, appear to provide the 'most accurate results and hence seem 
appropriate from an empirical perspective as welX as a theoretical one. On 

* the other hand, there is some question regarding thei£ s&ability, although the 

* - j - 

comparatively small amount of Scale drift associated^! tli»mo|t^of the IRJ" "~ 

* A V • - 1 * '* * *$- - - • * V. *' * 

* calibration designs found by Petersen etal. (1981?' is evidence 'iif support of 

. r ? 'i f " 5 v * ; ' * - ' 4 

. their application to.<_paralIel forms* of aptitude tests administered to groups 
< thai are similar tA. ability,.-; trt addition,, it is impojrt^nt t9 note that the 
studies rSvietfed indlcat^ that at present the effects of differential seJjLa- 

er|c : ' ;' "■ ' :' : \ . .7' ' " • / 



. : - 6 - 



J 



1 



bility and difficulty of tes*t forms w ^nd Che effect of Che noti-equivalence of 
examipee samples do not appear to be completely understood. a 

i . 

.* * The purpose 4 of the. present study was to 4 examine the*' results of applying 

-* • " * . * 

the three-parameter 16'gis'tic , model to the* vertical equating of the ,PSAT/^1SQT' 

to the SAT. The results of the IRT equating method/were compared to tho9£^,-i t - 

• . • y • . / 

, obtained from the conventional equating methods (l/foear and curViline<fr) N 

trickily usfed to equate the tests.. In addition, the^goodness of fit of the 

PS.AT/NMSQT *nd SAT dau to the three-parameter logistic model was studied. 



✓ 

* t 



- * ° Study D3fcign 

Description of the Tests : * 

'Both the PSAT/NMSQT and SAT are composed of multiple cb/oice itsms. The 

1 • ' * \ v 

•tests differ in lengt^ and difficulty, the PSAT/NMSQT contains 65 verbal tfnd * 

50 math^^jiical items wfrereas the SAT contains 85 verbal and 60 mathematical . 

i teras. * 4 

The PSAT/NMSQT cqnsists of two 50-piinute sections. The verbal section 

contains % only 5-cho ice, items; *£he mathematical* section contains a fixture of 

4- and 5-choice items. Raw scores obtained on tne PSAT/NMSQT are most typically 

f* * 1 

transformed to sealed , scores on the* Colieg-e Boarn 200 to 300 scale via linear 

,y ' s ■ si..- 

equating methods, > For score reporting purposes, the Jfinftl digit of th£ score is 
dropped; PS^T/NMSQT raw scores are actually formula scores generated from 

- \ - . ' * * 

number-Tight scores using a correction for guessing. • Raw scores are computed by 
the formula R - kW, where R is the number of correct responses, W is the number 
of incorrect responses and k =^i/A-l, A being the number: of choices per idem. 1 



, The SAT and its companion tfest, the Test of Standard Written English 

. , • ' \ ' - ■ > » 

(TSWE) ,' consists of. *six 30-rainute sections: two verbal sections, two mat-he- 

v * ' ■ ^ ' % ; ' * . 

matical sections, one TSWE section, and one, experimental section containing an 

>*',«' 
equating test or. pretest. The two' verbal 'Sections, one mathematical section, 

and the TSWE contain 5-choice items; the other mathematical , section contains a 

mixture of 4- and 5-choice items* Raw scores on the SAT are also typically 

transformed to scaled s.cores k Qn the College Board 200 to 300. scale by linear 

equating methods*. \his scale is retained for score reporting* SAT raw 

^cores t afe formula scores incorporating the correction for guessing procedure' 

previously described* ... ' 

It Should be noted that only the 'mathematical sections of Che PSAT/NMSQT 

and the SAT v were used in this stud£^- This is because the discrepancy in level 

of difficulty between the PSAT/NMSQT and SAT mathematical sections is greater, 

<?* 

than the discrepancy in level of difficulty for the corresponding verbal sections 
Thus,' the equating 'of the mathematical sectigns is the most. Representative of a 
vertical 'equating situat-ion. ^ - 

Equating Desigb * * * . * ' 

The PSAT/NMSQT Is e'quated' to^ the SAT using what is, commonly referred to 
as an internal anchor test design* (Angoff, 1971)* This design reqtJT^ admin- 
istering one form of the total test to one group of ■ examinees, a second form to 



a second group of examinees and a commpn set of item§ (anchor test) to both 
groups* The anchor test may be included within the total Best (internal anc 
or it may be administered separately (external anchor)* The anchor test is 



*IRT equating methods (curvilinear) were introduced in January, 1982. 



. v * - 8 - 

constructed- to be a miniature but otherwise parallel .version of the total test 
and is used to assess -the relative ability of the groups of examinees taking the 

two forms of the tes^f to be equated, 

* . * 

Standard practice in equating* new forms of the PSAT/NMSQT is to equate 

each new form of , the test to two old forms of the SAT through separate sets 

of' common items* One can imagine each of the two new PSAT/NMSQT forms produced 

annually as being composed of three sets of items: (1) items unique to" that 

form; (2)' items in common with one old SAT form; and (3) items in common with a 

second old SAT form. Tt is important to note that both new forms (Form l f and 

Form 2) of the PSAT/NMSQT share items in common with the same two old SAT forms. 

However, there exists no item overlap between the -fefeo new forms, i.e., each new 

\ • \ 

,fofm is equated back to the same two old SAT forms bkt through different sets of 

common items. Final scaled scores are determined, forWch of the PSAT/NMSQT new 

forms by combining the results obtained from the equatmgs to the two SAT old 

^orms,. In order^ to permit an examination of the individual equatings, results 

were not combined for this study. 

i 

Figure 1 contains a schematic diagram of the design used to equate the \^ 
PSAT/NMSQT to the SAT. As mentioned previously, PSAT/NMSQT Form 1 and Form . 

2 are alternate forms of the PSAT/NMSQT, each containing a subset of items in 

o J 

common with each of fhe SAT old forms (hereafter designed SAT First. Old Fqrm 

* » i * » 

\ and SAT Second Old Form). Equating samples for all methods coatatned approxi- v 
mately 3,000 cases selected randomly from data obtained at' the regular adminis- 
tration of each of the old and new forms shown in Figure 1. A total of four - 
random samples, one for each of the PSAT/NMSQT new forms andj one for each- ^ 
of the SAT old' forms, were selected. Table 1 presents sample raw secure summary 

- " ' ■> 

, ; 



PSAT/NMSQT < 



SAT 



Form ,L' 
Math • 



First! Old Form 
Math Sections 




Second Old Form 
Math Sections 




First Old Form 
Math Sections 




- Second Old Form 
, Math Sections 



Figure 1: Schematic Diagram- of Design- Used 
in Study for Equating PSAX/NM^QT < % 
Form 1' and Form 2 Math to SAT 
First and Second Old Forms. 
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Score Summary Statistics 


for Equating S^mpl 


1 

es , * 








9 

Total 


Test » 


* 4 
■# Anchor 


TesiSj 


Total Test/Anchor Test' 




Form • 


N 


\ Mean 


S.D. 


Mean - - 


S.D. ' 


Covariance 


* 

: First 


PSAT/NMSQT 
Form 1 


i > 
3367 


19,82 


11.27 


7,79 v * 


» 

4.92 


-' 51.58 


Equating 


SAT First 
Old Form 


3188 


20.99 


12.93 


8.12 


5.32 


'64.22 


Second 


PSAT/NMSQT 
Form 1 


3367 


19.82 


11.27 


8.30 


~m ■ ; 

4.84 


50.57 


Equating 


m SAT Second 
Old -Form 


2763 


" . 24.47 








* * 

* ' 61.34 


Third 


• PSAT/NMSQT 
Form 2 


3078 


22J62 s 


11.71 


8.76 


5. 05 . 

l . 


55.34 


Equating^ 


» 

SAP First i 
Old "Form 


, 3^88 


% 20. 9^ 


12.93 


8.26 


5.01 


* 

* 60.04 


* 

Fourth 


PSAl/NM^T 
% Form 2 


. 3078 ' 


22.. 62 


11.71 


8.93 


4.74 


;51.60 " . 

- 0 


Equating 


SAT Second 
Old Form' 


2763 


24.47 


13.18 


9.31, 


4.87 


59:3£ t s ,V- 



1 ' ' ' * 

The PSAT/NMSQT contains 50 items; the SAT contains 60 items; all anchor tests contain 20 
items with the exception Of the anchor test used for the ~f ourth equaling which contains 
19 items, 



.' . . A 

) ' ■ ■ • 
1 12 



statistics 'for the* total, te^fcs and anchor tests for the four *equa tings, * 



ft 



parameters 



described in Figure. 1. 

\ „ 5he"TDLne^r "equating raethcSts used in .this study both produce an equating 

transformation of the form. T(x)|f» Ax +^B, \rtiere T is Jthe equating »trans;f or- , 

> ■ ♦ ' • ^, % • ' ' ' ' - • 

mation^ Sf,is the test score- 1 to Which *~Lt*-is applied, and A snd B are. 

' \ % vi ' \ " . . 

estimated from the data. The TucHes and Lavine Unequatfiy Reliable linear 

• • * V- * * ' . ' 2 : ' 

equating -models (Angoff ., 19/1, pra. 579—583}. were 'used in 'this study. These 

1 * * ' - . ■ - 

models are based pn univariate selection sampling theory. Scores on the 

relevant selection attribute (the attribute on which the equating ^samples 

vary}. are as*sumed_j:6 be collinear with scores on the anchor test in the case 

of the Tucker model |wid with true ,score$ on both the anchor test an,d t;he ^est 

form in tfte case of Jthe Levine model. Scores on the anchbr test , are used to 

estimate performance* of the combined group'of examinees on both* the old and # 

new forms of the test; thus, simulating by statistical -methods J the situation 

in tohich the same group of examinees take both form^of the test. 

*• * * , 

The parameters A and B'of the equating transformation ar£. estiMfted by 

means, of an equation that expresses the, idea of equating iit standard .sccjre 

rijtjnus r ■ ^ — — - -* — * ; t ,t ~~ — — , — vx^ * "v - — ^ — ; — 



• - / >. ;-<ac - (y - M )/S " ,' (1) 

where » and y refer to the test scores to be equated, and M amd S refer to the * 

means and^stafcdard deviations of the scores m some gtroup of examinees. "Methods 

using the above equation differ* in their ^identification c^the' means and standard 

deviations to be estimated, fhe Tucker method *is based on the estimated means 

and standard deviations of^observ^'d spores for the fcorabine.d group whereas the 

Levine Unequally Reliable method is based qn the estimated m^ans and standard 

2 The Levine Unequally Reliable model was used for the PSAT/NMSQT Tarm 1-SA^ 
Second Old Form equating. All other equatings* employed the-Tucker linear 
model, " 1 O ■ * * 4 



- 12 - 



* / 



deviations o.f true scores for the combined, group- The formulas for computing 
the A and B parameters for the Tucker and Levine Unequally Reliable models 
are given in Figure 2. * *. \ 

The equipercentile model Inaintains that scores on two tes!t ^J^L are 
equivalent if they correspond to' the same percentile rank in some group of 
examtrtees. The procedure involves equating scores qn each teat form fo the 
anchor test separately withineach group. Scores on the two. fopns to be , 
■equated are then said to be equivalent i*f th§y .correspond to the same score on 



the anchor test. 



9 



'Finally, JRT equating yodels characterize equivalent 'scores op two test 
* forms as those scores which correspond to t^he same estimated level of the 
latent trait, ability, or skill underlying both tests. Item response theory 
assuries that a ^mathematical function relates the probability of a, correct 

. ■ ' ' ' ' "1 

response on an item to an examinee's ability (Lord, 1980). The mathematical 

function (IRT model) employed in, this study was the three-p s arameter logistic 

» * - s * 

, f * model. The model states* that the probability of a correct response to item i 

• — x: ' • » • 

1.7a i (e-b* i ) 



T- — ' 



P 1 (J9} =* c ± + (1- Ci ) 



1.7a (0-b.) 



(i=l,2,...,n) i f .: (2) 



1+e 



9 

ERIC 



where, a^, b^,' and c^ are three parameters describing the item and 9 ^ejlrasents 
the ability level of an examinee. ^ 

The item parameters and examinee abilities for the study were estimated 
*" using the program LOGIST (Wood and Lord, 1976; Wood et al . , 1976). ' The estimates 
are obtained by a (modified) maximum likelihood procedure which has been adapted 

~ # ' ' 4 

to accommodate omitted 4tem6 (Lord, 1974). The following constraints' wete * 



14 



* 

* * 

- 13-- .... r ^ , 

. . - / 


* 

» 


Tucker ' ' ' * 


': 


A = <s 2 + c 2 u (s 2 - s 2 j/s* As 2 + c 2 .'(s 2 - s 2 -)/s 4 )" 

yb * yvb vc vb- vb xa xva; vc va va 

** . * • 




B=-M,+C , (M -M^)/S 2 U -AM --AC »(M - M )/S 2! 
# • yb yvb vc vb vb xa xva vc ya' va 


, ■' ■ ". 


• ^ - - - ' * r - — . . 

* Levine Unequally ReliaSle / * 




. *t ^ _ 

• A ■ « S yb " S y"b )/iS vb - S l» b » h « S L r &.> /( & " 

- : . . * ■ ' - s 


» * • » 


* 8 ' V +> (M va " M vb )((S yb " S y f 'b )/( 4 " S l%^ " ^xa" 




J -Notation: _ *, - ■ 


/ 


> ^ — — ... * 
' New Test Form - X 
% Old Test Form ' / ,Y " J ^ .■ 
Either New or Old Test Form P . 
* Anchor Tese V 
^ \ t Observed' Score -x, y, v 

Error Score - ' fc « x", y"*, v f - f 
. * Group Taking Test X and Test V a 

„, 'Grou^ Taking. Test Y and. Test V . ' b - 

' * : . Group Taking Test P and -Test„V g 

Combined Group / . c or (a + b) 
£~ 'j-^^m ^MeaoL_ . M 

m Standard Deviation * * V s * 
Covariance • * C 


* * 

f* 

s ^ 
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Figure 2: r'Formulas for Linear Cohv£tsiqn 'Parameters 
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imposed on the estimation,, process: a's were restricted to values between 0.01 
and„1.75 inclusive; 9 T s were Restricted to a range of -7.0 to 6.0, and c's were 
.Restricted to values between 0.0 and .20. Additionally, each examinee was # 
required to have responded to 17 items in order to insure stable 0 estimates. 
The cdnstraints were- chosen on the basis of previous experience with the data 

- • 

i * 

and vfere imposed to speed convergence of the ^Likelihood function and to maximize 
stability of the- item parameter and ability estimates.. 

'Although a variety of equating techniques exist once an IRT model has been 
chosen, gnly estimated tjue formula score equating, (Lprd, 19>80, Chapter 13) 
was used for this study*. Estimated true formula scores £ and n on two^tests 
measuring the same ability, 6, are related- by the equations, 



n - I P-(6) - 



J-l- ' * 



n 

S- Q-(6) 
1-1 , 



s Q,(e) 
3-1- 3 



/A-l, and 



/A-l , 



(3) 



where, A- is the number of choices per item, and Pj( 9 )» represent the 

probability of a correct response for items -i and j as they appear in the . 
two forms to T>e equated and Q^(6), QjX e ). equal 1 - Pi(^) and 1 r Pj(9), respec- 
tively. Using expressions 3 and 4, it is possible Xo find an estimated true 
formula score g corresponding to an estimated "true' formula score n for any ~ 
given Q. ' ' r * 

Expressions 3 and 4 will not provide equated estimated true formula 

* * r 

scores for spores on the two test forms of interest that-fall below %he~ chajica. 

* j 

score level. Seyeral ways fexkst for dat^afcmining the relationship in. this "\ 
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f * • • 1 ' 

region. Kolen (1,981) use.d linear - interpolation. The method that was used for 

V 

this study involved estimating the mean and standard deviation of scores below^ 
the chance score level for the tfoo forms of interest and using" the estimate^ 
lvalues to establish a lineax relationship. 

The means and standard deviations of below chance score level scores were 

• • (. 

• - • s * v 

estimated using the following expressions: 



y 



n 

A x n 

^ • vrJi^'^i' and - 



(5) 



l c - E c 
i=l i=l 



where, ^ 



M » the mean of PSAT/NMSQT 'scores below chance level, 

- x 



S ■ the variance of fSAT/NMSQT scores below chance level, 

x '% * 

A =* the number -of. choices per item, and 

" " - : n . 

c . * the psuedo guessing parame'tes* for item i. 
i • • . k * 



(6) 



Equations 5 and 6 were repeated to obtain M an d s y , the estimated mean 
a$d. variance of below chaYice level scores foir the SAT old form of interest. 
Linear parameters for equating PSAT/NMSQT scores below chan'ce level to SAT 



scores below chance level were determined £s follows: 

• * S ' ' 

- • X ' 



B = M -AM 

y x 



(7) 
, (8) 



The linear parameters (A aV- B) are used to form* the following expression: Jt 



score (SAT)' * A [score (PSAT/NMSQT) ] ' + B 

* * v ' 



(9) 



ERIC 



t - 



7 



^ ' The item calibration plan for the IRT equatitigs" is illustrated in Figure 3* 
The entire matrix showr/ in this f igure4x;epresents a single LOGIST run. Each bf 

tHe four* groups £s conceptualized as taking exactly thS^ame te^st. The test is 

"A /• \ • ' , 

. conceptualized as containing eight .components designated by the column headings* 

Mr ■ * 

ThiSi design, places all item parameter estimates on the same §c&le and permits 

true formula score equating of each PSAT/NMSQT-SAT pairing shown in Figure 3* 

- \ > ' '* ' * 

uie results of the equatings were evaluated singly by comparing the raw-'/ 
. \ • • . f % - . 

score to scaled-score transformations (tabled data. and graphs) obtained by the 

three equating methods** Unfortunately, no objective criterion is available in 

•this study to judge the adequacy of the equatings. Most prpbably the asgAirfip- * . 

tiorts underlying all of the models have been violated to some extent and thfeir 

robustness in an anchor test situation is not clearly^ understood* . (See, „ however, 

M^i^o, Petersen and Stewart (1979) i&nd Petersen, Marco and Stewart (in ptfess) 

for a detailed analysis of the robustness of many anchor-test design methods*) m 

In the absence of- a true criterion for judging the equatings, an effort was made 

to examine the goodness of fit of the data to the IRT model. T|& toethod used 

x « * 

for this examination is described in the following section. ^ 

«• 

Assessment of Goodness o>f Fi.t ' . ■* 

Researchers^ often attempt to assess the fit of an item response theory 
model to real data using a chi-square test or other similar approaches '(fright 
and Panchapakesan, 1969; Wright and Stone, 1979)* The problems associated with 
this approach have been discussed extensively in the literature (Rentz and 
Rentz, 1978; Divgi, 1981; Rentz and Ridenour, 1978; McKinley and Reckase, .1980). 

* 4 - 

These problems have both theoretical and practice implications. From 4 theoret- 
ical point of view.a problem exists in that ch^-square tests require expected 
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Figeirer 3: .Calibration Plan- foif! IRT Equating of , P&AT/NMSQT Form 1 and: Form 
.\TheT entire matrix rejfteareats *a."single w calib ration run. Crosses 
. grbu.#s were actually exposed" to ^ Each PSAT/NMSQT and SAT sample 

6ases. * . . • »' 



2 Math- to the two SAT .old Forms, 
indicate iteitfs that' examinee ^ 
contains approximately 3,000 



values dhat are available only ( when the parameters" of the model ( 9^, ai> b^. and 

c^, in the case of the threevparame'ter .model) are known; in actuality, we haVe 

oply estimates of th6se parameters. These estimates are likely to behave 
* * 

differently from the known or true parameters in a statistical test* The _ 
practical problems are related to the interpretation of the cfti-squa^e values 
and their associated probability levels, these problems will be discussed in 
subsequent sections of fchi^ paper. One alternative to the various chi-s'quare 

tests is the use of a graphical technique which involves \he comparison of the 

• • . * - . 

regression of the observed^ proportion of people getting an item* correct on 

w « . « 

estimated 6 (empirical regression) with the item response function based on the 
'estimated item ^parameters (/estimated regr$s$jton)' (Haqbleton, 1980; Stocking., 
-1980). The resulting plots. are referxed to as item ability regressions* 

' The problem *with using item* ability regression plots Go assess goodness bf 
fit is that the process if » fairly subjective. The' authors found it quite diffi- 
cult to, examine 141 graphs Con§ for eaclv item represented £n Figure 3) arid make 
consistent judgements regarding the goodness of fit of each item. For this 
reason, it was decided, to use a fit statistic leading t^o a chi-square like test 
in conjunction with the item ability regression plots* It-. should be emphasised 
that the statistic was used only to aid^ in the interpretation of the plots. No 
specific meaning was attached to either the size or the probability levels of 
the values obtained from the application of the statistic, a The fit statistic 

As ; ' 

and *the item ability degression plots will each be described briefly in^the 

/ \ s * ' V 
remainder of this section. * - ' 

: • ) - 

The Fit Statistic , . 

Thp fit statistic, referred to as Q^, is based on a statistic, Qj, 
suggested by Yen (1981), T^etwo .statistics are very similar-, the basic 
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difference being the manner in which examinees are grouped into cells based 

. * ■ O ~* 

upon their ability estimates. For both statistics, the initial step is* to 

♦ » 
rank order examineea abilities. For Q^, examinees are divided into 10 cells 

— with approximately equal numbers of- examinees in eabh, cellL For examinees 

are divided into 17 cells- as fellows. Examinees are placed into 15 equally 

0 

spaced intervals -for*9 between +3 and -3. Thos,e' examinees with 8 greater than 
+3 are glaced into a si^le cell and examinees with 9 less than -3 are placed 
in another cell. Should^ny cell contain less then 5 examinees, it -is collapsed 
/'with the adjacent cell closest to Q =» 0. ^The only ^remaining difference between 
the tv/o statistics is* 
a particuj 

notation,*^?e value of rthe fit statistic for item i is - 




%4xJf* 9 the observed proportion of examinees in 
ell is adjas^l fof examinees omitting the item. Using Yen's 



where, 



7 



17 v°n - e h )2 



(10) 



(1 - B iJ> 



N. is the number of examinees in cell j, 0^. is the observed proportion 
of examinees in celT^ that passes item i (adjusted for omits) and, 
E. . is the predicted proportion of examiriees in cell j that passes 

item i, . m * 

N. ' •' 



(ID. 



* o 

RJC 



} 2 



- . ~ ~~ \ * \ - • . * \ \ 

whejre P (8, ) is the item response function (equation, 2) for item,i. It should 

» . * j / 

be note^ that the summation is over examinees in cell j. The degrees of 

J; • -\ 

freedom are the number of independent data points ,(?ells) less the number of 
item parameters estimated from these, data points. The number of estimated ' 

it,em parameters is not three in Al. cases, - In.sotee iustindes the value of 

' . ~ * * 

the iteiji discrimination parameter (a.') vfes set to the upper bound for ; 

f * i * * » » * ... ** # * 

3 * # ' n ' ; % 

a values. In othetf. instances the value of the .psufcdo-guesslng parameter 1 



(c ) was se,t to 4 common value. ^ Fit statistics were determined 

1 

using Q, for each of the 141 PSAT/NMSQT and SAT items used in this Study. 

)• 

Item Ability Regression Plots , i 

> ■ * , 

The'item ability regression plots were obtained as follows. The ability 

scal^ (0) is subdivided into 15 equally spafced intervals for arrange of -3 to * 

+3. For each interval, equation (12)' J.s used to compute' P the proportion of 

people in interval j respondirig^porrectly to item i (adjusted for omits). That 
\ s , : N* + N.°,/A . 

1S> ^ . P = ij y ij , where ' (12)' 

is the number of. Examinees in the jth interval responding correctly 
J to item i, , 

ft® . is the ^number of examinees in the jth interval tjhat omitted item i, 
• A is the number of alternatives per item, 

$j a is the number of examinees in interval j that reached item i. 



Upper atid lower bbunds were svt for all item discrimination, parameters 
to prevent the estimates from becoming unreasonably , large or small. 

When LOGIST determines it cannot acurately estimate the* c parameter for 
a certain item, due to insufficient information at lower ajxflity levels, 
it uses an, estimate of c obtained by combining 1 all such -ixems; for this 
study, g for 5-choice and 4-choice items .falling into this category was . 
.067 and .123 respectively. 
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f For each item? 15 P's are plotted as squares whose areas are proportional 
to (these values constitute fhe empirical item ability regression). Als^o - 

plotted with each^ square is a line of length 4 /PQ/N^ , where P and Q are computed 

> 

from the estimated item response function. The resulting 15 lines are centered 

> / 

on the estimated item response function which also' appears on the plot. It 
should be noted that although the line is^a trough estimate of the x ,95 confi- 
dence interval around the item response function^ it is not being used as a 

statistical test for several reasons: 1 (1) the use of 2 as a coefficient instead 

* 

of 1.96; (2) the use of the inappropxiate symmetric normal, approximation to the 
binomial confidence interval around -4:he response function (particularly a 
problem for extreme values of P); and (3) the use of an interval based on * 
estimated item parameters. Item ability regression plots w£i;e obtained for 
each of the 141 PSAT/NMSQT knd SAT items used j.n this study. 

Results 

Equating^ # - 

Tables 2-5 list the raw-score to scaled-score transformations for each 
PS AT/ NMSQT-S AT pairing for each equating method.^ The tables also include 
discrepancies for the linear and equipercentile equating results as compared ' 
to, .the results obtained from the IRT equating. The discrepancies were computed 
by subtracting the scaled scores obtained using the traditional equating methods 
from those obtained using" the IRT method. The data summarized^ in Tables 2-5 are* 
presented graphically in Figures 4 and 5 t _ Figure 4 contains plots of the 
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TAQLE 2 



£, w - PAW SCORE TO SCALED SCOPE TRANSFORMATIONS AND RESIDUALS 

' PSAT/NMSQT FORM" 1 TO SAT FIRST OLD FORM 
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PAk SCORE TC SCALEO SCOPE TRANS POP-MAT IONS AND RES IDVAL S 
\ PSAT/NMSQT FORM i TO SAT SECOND OLO FOPM 
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RAK SCOPE T« SCAL60^SC0fE TRANSUDATIONS AND tyESIOUALS 
'OSAT/NHSOT' FORM ? TO SAT F^PST OLO FO»M 
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Figure 4: Plots of raw score to scaled score transformations resulting from 
application of the three equating methods* ^ 
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Figure 5:« Plots of differences between converted scores '(IRT-conventioiiaJ. method) 
resulting from application of three equating methods. 



raw-score to scaled-score trans forma ticms obtained for the four equatings. Each 

- / ; * " ' * 

plot in tliis figure, compares the results of t&e conventional equatirtg, methods 

* / " r ' " ' * ' 

(linear and equipercentile). with the result^ of the IRT equating. 

Plots such "as those shown in Figure /. tend to Emphasize the similarities 

between the equatings rather than the differences. Plots of residuals are often 

» . • '.,../ 

informative when used in conjunction with plots of raw-score to scaled-score 

*~ / * * 

transformations. Figure 5 contains residual plots for the four equatings. fhe 
IRT equating was used as' the coiiipatison equating and the difference between \ 
scaled scores obtained~by each of the conventional methods* and the IRT method 
was platted against raw scare» > 5 *• * 

> 

Finally, the »ef feet on scaled score summary statistics of each equating 

■ <* # 

<> 

"method is presented in Table 6. « The data" presented in Tables 2-5 were used* to 
compute these statistics. The frequency distributions giveiv in fables 2-5 ^aee^ 

--- t 

simply a convenient vehicle for converting the scaled scores into interpretable 
summary statistids. Any reasonable frequency distribution would suffice. The 
distributions selected were the PSAT/NMSQT Form 1 and Form 2 populations. 
Examination of the data fouiyi in Table 2 and .illustrated in Figures 

- s : 3 

4 and 5 indicates closer agreement^ (for scores above a formula scor y e of 10) " 

* • * * * * * A 

r rt 

between the IRT and equipercentile methods than the IRT and linear methods /' 

for the PSAT/NMSQT Form 1 - SAT First Old Form pairing. The IRT method tended 

to yield higher* scaled scores at the upper" end^ of the score scale than^ either of 

' ' * 

the two conventional methods. The IRT transformations v are si ightljrl higher than 

f* 

the linear • transformations at the lower 'end of 'the score scale and have a % - 

« s . "\ ,/* * ■ . 

tendency to be lower than these transformations through the potion of the score 

range where the largest number ,jof obtained scores occur. Fairly close agreement 

between the IRT and equipercentile equating is observed frir the middle portion,, 

- s 

• . /t 
* * / 

-A 



• - 29 - * 

'-Table 6 % 

Scaled Score "Summary" Statistics 'Resulting fr-tom 4 ** ^ . 
Application of Three Equating- Methods • 



Form 
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Mean S.D. " ' Mean S.D. 
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$ 


45.53 


11.41 

* 


t 

45.52 


11.38 
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t * 
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' 9 *• 
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Toutith 
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SA.T Second 
Old Form 


462551 , 


45.90 


11.09 


46.27 • 
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of tha -score V&nge. Noteworthy' discrepancies occur at the lower end of the < 

y * * - • * 

score scale where the equipercentile method yields higher scaled scores tTYan the 
IRT inethoa\ ' % , ' 

The differenqes between the raw-Score to scaled-score transformations 
resulting from the different equating methods applied 'to the PSAT/NMSQT Form 1 - 
SAT First Old Form pairing are reflected in the summary statistics -found-in 
Table 6. Scaled score means for the IRT and equipercentile methods agree fairly 
fclosely. Mean scores resulting^f^om the linear method are slightly higher "than 

those obtained using the IRT equating, reflective of the lfewer *IRT transfor- 

. 

tmations obtained for the major portion of the: score reporting range* - 

Table 3 and Figures 4 2nd 5 contain informatibn pertaining to the PSAT/NMSQT 

Form 1 - SAX Second Old Form pairing. Similar to the results of the equating to 

the SAT First 01d.*Forin» the IRT method tends- to ~yield higher scaled scoters than 

either of the conventional methods for the upper end of th^ score scale. The 

« ♦ * 

IRT scaled* scores are again lowet than .the linear scaled score** for the mid- 

, " " ♦ ' . \ * ' " • 

portion of the score ijange which contains the majority of observed scores. IRT 
t '» ( t * 

transformations do not agree as closely with the linear transformations obtained 

for lower scores as they did' for $he equating to the SAT First Old Form. In , 

this case the tI|T transformations are somewhat higher in relationship to the 

linear transformations as compared to* thfc first situation. There is a slightly 

greater discrepancy between the IRT and equipercentile transformations for this 

equating than that found foir the equating to. the SAT First" Old Form. In this ' 

instance, the IRT method tended to produce scaled scores that wfejre consistently 

» 

lower than those produced by the -equipercentile method for. all scores except 
those at the upper end of the Score range. * . • 
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.Examination of .the summary statistics for this equating Contained in Table 
6 indicates a closer agreement between the scaled score m$ans obtained from the 

* * , / . 

linear and equipercentile iqethods than the /Scaled score mean obtained from the 

* * - «- * 

IRT method and either the linear or equipercentile method. As was the case 1 
— * * i 

with the SAT First Old Form pairing, the IRT scaled score mean is lower than 
that obtained from either the 'linear or equipercentile methods, 

• Information relating to the equating of PSAT/NMSQT Form 2 to the SAT First 

Old Form is summarized in Table 4 and Figures 4 and 5,. Once again, it can be 

noted that the IRT transformations* are higher tharTeither the linear or equiper- 

centile transformations fo.r the upper end, of the scare scale. It appears that 

s« * 
for the mid-portion of the 1 score range the IRT transformations -are lowe# than 

the' linear transaminations but not for as great a range as pteviously observed, 

IRT trans formations become higher than linear transformations ,at a point where a 

large number of .observed # scores are fctill accurring and continue to be higher' 

than the linea'r transformations to the bottom of the distribution of observed 

scores. The IRT transformations agree fairly closely with the equipdrcfcntfle ^ 

transformations with the exception of the upper end o£ the score range, 

• Reference to the data^^r this equating contained in Table 6 indicates 
-that for the first time the IRT scaled score mean is slightly higher tha,n that 
obtained by the iinear method. An interesting point to note' is the close agtee- i 
ment between the scaled" score mean^ obtained from all three of the methods. 



Table 5 and Figures 4 *and 5. summarize the results of the PSAT/NWSQT 
Form 2-SAT Second Old Foym equating. The typical pattern of the IRT equating 
"resulting in higher scaled scores for 'the upper end of the distribution is 
observed, however, this discrepancy is not as great as previously observed for 
the IRT-equipercentile comparison* The IRT method results in lower 'scaled 
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* 



scores for the mid-portion of ( the score range\when compared to scaled scores 

? 

, obtained by the linear "method Equipercentile scaled scores a^e higher than 

* ♦ - - "* 

j M.I , * , 

the IRT scaled scores for all scores except those in the very upper portion of 
the score range* As indicated by the data for this equating given in Yable 6, t 
the IRT scaled score mean is lower tt\an either'^EIie linear or equipercentile * 



scaled score mean. 



To summarize, in general, IRT equating, when compared to linear or equi- 
percentile methods, yields higher scaled* score values for the highest raw 
.scores and to some^extent, (particularly -when compared to th$ linear equating 

\ results) yields higher scaled score values for the lower faw scores- In the 

* v » * . * 

mid-portion of the score range IRT transforations tend to, be slightiy lower 
than those obtained by linear or equipercentile methods* 



^Assessment of Goodness of Fit 



The results of the goodness of fit analyses are presented in Table 7 

> * v' 

and Figures 6-31. Table 7 contains the overall value of the chi-square 
statistic as well as the contribution to this statistic of each of the JL1 
ability level intervals for the 141 items. Inspection of Tthe individual 

contributions to the overall value of for an > 5^m is important because in, 

*' *** * » 

some instances only one interval may contribute close to, or over, half of 
the total*'chi-square value.. -This situation generally occurs for intervals 
at the extremes of the ability continuum where the number of examinees in 
the interval its small, thus 1 only a few^deviant response patterns* may cause- 
the interval to contribute greatly to the total chi-square value. Under 
these circumstances, simply-using the overall value to assess the goodness 
.of fit of the item would be inappropriate* , . # * 
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Table 7 (cont'd) 



.Item 
Parameters 

ft 

C 



fT 

<< 

o 
o> 

ri- 
ft 
OQ 
O 

T* p. 

(D 

CO 

I 



. 2.8 

. ?** 
2.0 , 

1.6* 

1.2 , 

;» 
.4 

.o 
-.4 

n 

-1.2 
-1 .6 
-2.0- 

-2;« 
<-3. 

TOTAL CHI 
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3.3<U5 
3.S877 
3.1547 

► 0.13 ♦ 

- O.IH - 

- 0.43 ♦ 

- 1.12 - 

- 0.48 ♦ 

► $.9* - 

► 1.26 - 

• 0.74 - 

- 3.?5 ♦ 

- 0.06 - 

► 1.78 - 

► 2.42 ♦ 

► 0.08 ♦ 

• 0.0ft * ♦ 

• 3*26 

• O.Oo J 

► 0.2ft «• 

23.33 



?6 

0.O420 
0,704? 
0.2230 

► 0*1* 

- 0.26 

► 1.47 

- 0.3* 

► 0.35 

- 0.00 

- u.3l 

- 3. rt 7 

► C.23 
• 0.05 
' - 0*i)l 

► 1.30 
" i.2l 

- 0.2? 
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5.4? 
0.14 
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36 
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1.3525 
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0.0251 
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0.06 73 
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4- 
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4- 
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* 0.0<J 




0,51 
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♦ 
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0.31 
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4- 
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0.09 
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0.00 
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0.49 
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4- 
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4- 
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4> 
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C.02 


♦ 
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4- 
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♦ 
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2.66 
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4> 
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4> 


0.26 


4> 
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0.02 
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4> 
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1.50 




2.43 


4> 


0.76 
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0.01 


4- 
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2.64 


4- 
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4> 


4.57 


4> 
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4> 
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4> 
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0.65 


4> 
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•- 1 .6 1 
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2.64 


4- 
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2.33 


4> 
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0.03 


4> 


5.16 


4> 
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4> 
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2.53 
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- e.e* 


4> 


?.53 




6.6? 


*\ O.OO 
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0.20 


4> 
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0.05 


4> 


2.81 


4> 


0.05 




0.62 


• 4 , 


- 0.?7 


4> 


0.7?. 




0.04 


- * 0.34 


4- 


0.71 




3.00 


4> 


1.11 




I • 43 




1.59 


4> 


1.95 


4> 


1.18 




0.05 , 


.0 


«•* 3.?5 


+ 


3.70 


0.13 


4- 


1.69 


4- 


0.34 




1.83 




0. 38 




7.8? 




0.?5 


4> 


1.47 




0.O5 


4> 


0.*7 


~* -.4 


♦ 0.54 




U*3 


4> 


9.65 




2.36 


4- 


2.29 




0.27 




1.45 




4.23 


4> 


0.62 




0.04 




0.13 




0.15 


-.8 


♦ 0.46 




0.42 


4> 


0.30 




0.30 




0.34 




0.01 


4> 


1.27 




0.96 


4> 


0.45 




6. 94 




4.21 




0.03 


-1.2 


- 1.3ft 




0.04 




0.01 


4> 


0.65 




2.10 




3.00 


4> 


0.59 


4> 


8.44 


4> 


0.01 




3.41 


4> 


2.?7 


4> 


3.29 


-1.6 


- 1. 00 




O.CO 




8. 16 


4> 


0.02 




3.65 


4> 


4.36, 


4> 


3.30 


4> 


10. ?9 


4- 


0.33 




0.02 




0.15 




0.41 


' -2.0 


- 0.87 




6.04 




1.45 


4> 


0.10 




o.oo 




0.04 




0.78* 


4> 


7.37 


4> 


0.08 


4> 


3.79 


4> 


2.33 


4> 


0. 12 


-2.4 


- 0.72 


4> 


0.03 




0.38 


4> 


a-^o 


♦ 


0.05 




0. 79 




0.02 


4> 


0. 78 




J 


4> 


2.96 


4> 


4.68 


4> 


0.01 


-2.8 


- 3.27 


4> 


6.06 




0.15 




0.39 




I,. 41 




0.76 


4> 


4.09 


4> 


1.09 




1.17 




0.27 


4> 


0.12 




0.76 


<-3. 


- 0.04 


4- 


4.79 




0.00 


♦ 


0.20 


4- 


.4,01 




3.7} 


4> 


1.14 


4> 


1.47 




0.01 




0.70 


4> 


I .84 


4> 


2.25 


TOtAL Cfll 


15*18 




24.11 




41.79 




10.11 




25.13 




26.41 




16.37 




69.03 




9*56 




42. 8<) 




19.61 




22.56 
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Table 7 (cont'd) 



r 



, Item 
Parameters 



Item Numbers 



25 * 



* 

H 
H* 
rr 

o 
p 

rr 

oo 

. o 

Hi 



A 




3.3915 


8 




0.S877 


C 




0.1547 


> 3. 


♦ 


0.13 


2.8 




0.18 
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0.43 


2.0 




1.12 


1 .6* 




0.48 


1.2 




$.94 


;p 




1.26 


.4 


r 


3.74 


-0 




3.p5 


-.4 




0.06 


-.n 


¥ 


1.78 


-1.2 


♦ 


2.42 


-I .6 


*> 


0.08 


-2.0^ 


♦ 


0.08 


-2.# 


♦ 


0.26 


-2.8 




0.09 


<-3. 




0.28 



TOTAL CH! 



20.33 



26 

0.9420 
0,704? 
0.2230 

♦ 0*16 

- 0.26 

♦ 1.47 

- 0.38 

♦ 0.0* 
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- 0. rt 7 
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- 0.05 
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0.2? 
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- 0.00 

- o.ao 
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28 
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I . 6635 
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0 
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~¥ 
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2.50 
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♦ 


0.09 


¥ 


0.07 
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♦ 
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1.99 
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0.85 
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♦ 
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0.34 




0.01 




1.27 




0.96 


¥ 


0.45 




6.94 




4.21 




0.03 


-1.2 


- 1.38 




0.04 




0.01 




0.65 




2.10 


¥, 


3.00 


¥ 


0.59 




8.44 


¥ 


0.01 




3.41 


¥ 


2.27 


¥ 


3.29 


-1.6 


- 1.90 




0.00 




8.16 




0.02 




0.65 


¥ 


4.36, 


¥ 


3.30" 


♦ 


10.29 


¥ 


0.33 




0.02 


¥ 


0.15 




0.41 


-2.0 


- 0.87 




0.04 




1.45 




0.10 




0.00 




0.04 




0.78- 


f 


7.37 


¥ 


0.08 


♦ 


3.79 


¥ 


2.33 


¥ 


.0.12 


-2.4 


- 0.72 


¥ 


0.03 




0.38 




Q.40 


♦ 


0.05 




0.79 




0.02 


f 


0. 78 




0.42 J 


♦ 


2.96 


¥ 


4.68 


¥ 


o.oi ■ 


-2; 8 


- 0.27 




6.06 


¥ 


0.15 




0.39 


♦ 


1.41 




0.76 


¥ 


4.09 


f 


1.09 




ni7 




0.27 


¥ 


0. 12 




0.76 


<-3. 


- 0.04 




4.79 




0.00 




0.20 


¥ 


,4.01 




0.75, 


¥ 


1.14 


f 


1.47 




0.01 




0.70 


¥ 


1 .84 


¥ 


2.25 


TOtAL Chi 


15.18 




24.11 




41.79 




10. 11 




25.13 




26.41 




16.37 




69.03 




9.56 




42.89 




19.61 




22.56 



37 
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% Table 7 (cont'd) 



Item 
Parameters 



> 
cr 



o 

rt 
(3 

0 
»i 

03 



Item Numbers 



50 



51 



52 





0.7104- 




1.62*0. 


0.7*83 


0.6W8 


0.8215 


8 




I .6595 




2.1587 


-t.9031 




1.1465 


-0.5892 


C 




).0913 


0.1051 


0.0673 


0.1228 


O.I278 


' > 3* 




1.88 




0.10 


' ♦ 


0.32 




10.67 


4- 


0.06 


2.8 








0. )4 




0.05 


♦ 


0.17 


4- 


0.15 


2.4 




0.37 




1.42 


♦ 


0.17 


♦ 


0.60 


♦ 


0.61 


2.0 




1 .20 




0.95 


T 


0.34 




0.85 




1 .99 


1 .6 


♦ 


0*02* 


♦ 


0.50 


t 


0.14 




1.00 




0.09 


U? 


4- 


0.02 




0.16 




0.01 




1.46 




). 36 


♦ ft 




0.^0 




0.80 


1 


3.316 


4- 


0.O1 




0,77 




4- 


0.84 




4.58 




1.26 


♦ 


0.81 




1 .«0 


.0 




0.16 




0.34 




0«36 




1.35 




0.54 




4- 


3.11 


♦ 


0.1*2 




0.40 




o.4>r 


* 


7.96 


-.8 




0.83 


♦ 


1.61 


♦ 


1 0.97 




b\ 7 7 




0,30 


-1.2 




0.11 




0.14 


♦ 


0.29 


♦ 


0.16 




2.19 


-1.6 




0.09 


4- 


1.03, 




0.17 




0.51 




0.73 


-2.0 


4- 


2.5° 


4- 


6.00 




2.5? 




0.65 




4.06 


-2.4 


4- 


2.51 


4- 


0i 02 




1.81 




0.11 




0.39 


-2.8 


♦ 


0.00 




0.28 




\ 0.34 




1.80 




1.11 


<-3. 


f 


0.19 




- 0.3* 




0.12 


♦ 


0*11 




0.60 


TOJAI fHI 




15.09 




l*y? 7 




12.3* 








23.39 



54 

0,7496 
-0.0724 
0.0 



0.22 
0.51 
0.77 
7. 'LI 
1.77 
0.34 
0.86, 
1.59* 
3.53 
2*6:7. 
2.56 
0.01 
0.7*2 
1.67 
0.19 
0.3,5 
<0.09 



2 4., 9,5 

V* 



55 




56 




5T 




58 




59 




60 


0. 7927 




I .'033 1 


0.9?8l 




1.1047 


0.9603 


0.6932' 


1.6959 


0*1668 


- 


0.3775 


- 


0.1436 




t.6202 


-0.2895 


0.151 7 


0.4326 


0.0133 


0.1052 


0.1934 


0.0473 


" 0.14 


4- 


0.03 


4- 


0.05 


4- 


0.02 




0*06 


4- 


0,2? 


0.33 


♦ 


0.09, 


4- 


0.13 


4- 


0.07 


4- 


o.oo-* 




0.46 


0.03 




0,81 




0.37 


4- 


0.35 




0.28 


4- 


1.69 


• CO? 


4- 


0.29 


4- 


2.06 




0.24 


4- 


0.29 




0.99 


* 0*66 


4- 


0,Q7 




2.17 




0.01 


4- 


0.00 x< 




0.00 


0.36 




0.01 


4- 


1.78 




0.00 


4- 


0,08 


4- 


0.67 


• 1.07 


4- 


o.Ot 


4- 


0.19 


4- 


0.22 




0.50 




1.34 


" 1.02 




0.14 




0.99 




0.16 




0.00 


4-* 


-0.1* 


► 0.31 




0.01 


4- 


0.24 


4- 


0.21 


+ 


0.09 




0.14 


• 0. 19 




0.01 




1.06 




0. 11 




JI.07 


4- 


0.48 


• 0.01 




0.08 


4- 


0.31 




0.52 


4- 


1>56 




0.63 


• 0.44 


4- 


1.27 


4- 


0.05 


4- 


1.1 1 


4- 


I.4I 


4- 


0.87 


• 0.13 


4- 


2.39 


♦ 


0.68 


4- 


0.03 


4- 


0.18 




0.00 


> 0.27 


4- 


1.64 


4> 


0.21 




0.00 




0.04 




0.*5 


> 3.21 


4- 


0.03 




1.32 


4- 


0.42 




5.5T 




0.14 


► 0.46 




0.06 




0.09 


4- 


0.13 


4> 


2.02 


4- 


. 0.00 


- 0*12 


4- 


0.00 




0.17 


4- 


1.27 


4> 


0.05 




0.82 


8.76" 




6.94 




11.86 




4.92 




12.22 




9,00' 



Item Numbers 



> 
cr 



* rt 

rr 
O 
OQ 
O 



Item 












































71 


arameters 




61 




62 




63*" 

0 




64 * 




65 




66 




67 , 




68 




69 




70 




*- 




0,8078 




0*5542 




1 .0405 




0.4447 




1.2287 




0.8303 




0.726L 


0.7850 




1.2901 


0. 7770 


0.8*47 


8 




C.9CJ6 




1*7510 




0.8634 




0.6154 




0.3991 




0.1490 




0.2010 


0.1460 




1.5460 




1*2709 


- 


3.6691 


C 




0.1661 


1 0*0673 




0.06fc0 




0 .-06'7 3 


0.0959 


0.1510 


0.1195 


0.1999 




0,1228 


0.'12?8 


0.1228 


> ?. 


4> 


0.47 




3*39 


4- 


0.01 


4- 


0.01 




0.02 




0.16 


4- 


0.47 




0.92 


4- 


0.00 


4- 


0*05 


4- 


0.06 


2.8, 




8.39 




- 0*72 


4> 


0.05 


4- 


2.40 


4- 


0.08 




0.64 


4- 


1.25 




. 0.05 


4- 


4.00 




5*18 




4.13 


2.4 


4> 


1.95 




0*17 




2.33 


4- 


7.61 


♦ 


0.42 


♦ 


1.91 




0.05 


4- 


0,42 


♦* 


0.01 




3.65 


4- 


0.73 


2.0 


4- 


1 .63 




0*07 


♦ 


0.00 


4- 


3.60 




0.00 


♦ 


0.08 


t 


0.46 . 


4- 


Oil! 


•4- 


0.07 


4- 


0.49 


4- 


1.08 


1.6' 




0^32 




0*53 




5.72 


4- 


1.79 


♦ 


C.05 




1.55 




0.31 




2.27 




2.07 




2.30 




0.07 


* 1.2 


4> 


0.85 




0*46 




0.26 


4- 


1.62 


♦ 


rO.6? 




C.10 




0..02 


4- 


0.98 




'24.50 




0.7.3 




, 1.12 


.8 




2.04 




6*44 


♦ 


0.21 




0.*4 




0.25 




t.07 




0.03 


4- 


2.28 




0.01 


f 


0,02 




2,23 


.4 ■ 


4> 


0.'8 


4- 


0*C9 


«V 0.01 




2.48 




0.00 


4- 


0.34 




0.04 


4- 


0.04 




0.21 


4- 


0.04 


4- 


1.12 


.0. 


4- 


1.37 


4- 


0.18 


- 0.41 




1.56 




0.09 




. 0.39 


f 


0.09 




2.11 




0.45 


4- 


0.01 


4- 


0.69 


-.4 




4.61 


4- 


4.98 


f 


5.04 




0.70 


♦ 


0.03 


4- 


0.77 




0.03*-* ♦ 


0.01 


4- 


0.82 1 


f 


1.14 




a,i2 N 


-.8 




0.06 


4- 


1**M 




0.26 




2.06 




).00 




0.67 


4- 


0.88 




0.69 


4- 


1.10 


4- 


0.01 


4- 


3,47 


-1.2 


4> 


l.*3 




0*20 




3.40 


4- 


4.45 


4- 


0.00 


4- 


0.13 




0.50 


4- 


2.35 


4- 


0.66 




0.04 




2,51 


-1.6 


4> 


0.89 




0.04 




1.72 


4- 


ZAb 


♦ 


0.82 




0.02 




0.53* 




0.08 




1.46 




0.39 


4- 


0,01 


-2.0 


4- 


0.59 




1.05 




0.00 


4- 


3.25 


♦ 


0.01 


4- 


2.39 


4- 


0. 43 




0.22 




0.03 




0.38 




t.lJ3 


-2.4 


4> 


0.54 




1.78 


4- 


0.4? 




0.00 


♦ 


0.75 


4- 


0.52 


4- 


1.99 


4- 


2.00 




3^W 




0.42 


4- 


0.09 


-2.8 




0 V 36 




4.31 » 


4- < 


7.52 


4- 


0:11 


4- 


2.85 




0.95 




0.14 


4- 


1.76 




0.22 


4- 


. 6.50 


4- 


0.50 


<-?♦ 




1*81 




0.54 


t 


0.01 


4- 


0.61 


♦ 


*0.00 


4- 


1.^6 

/ 


.4- 


1.03 


4- 


3-56 


4- 


7.67 




0.09 




0.29 


-torn chi 




28*18 




26. IT* 




2*.^8 




34.96 




6.00 




13.56 




8.25 • 




19.86 




42.90 




15.43 




20.35 



7? 



0.9985 
0.0016 
0.-0867 



4- 


0.06 




i.a^ 




0.39 




1.59 




3.87 




0.42' 


4- 


Z.S3, 


4- 


o.or 


4- 


0.93 


4- 


0.49 




5.29 




2.23 


4- 


1.24 


♦ 


0.92 


4- 


(T, 1 1 


4- 


2.02 


4- 


4,14 




28.11 



: 1 



ERLC 



38 



- 36 - 



Table 7 (cpnt'd) 



Item- 
Parameters 

a ■ 
8 

c 



> 
cr 



J* 
rt 
*< 

o 

0) 
rt 
ro 

OQ 

o 
m 

CO 



Item Numbers 



7? * 

tf.6 759 
1/0616 
0.2148 



74 * 



7? 



76 * 



1. 3059 , 0.9747 „ 0.4008 
1.0332 1.9264 -0.4242 
0.0505 0.1001 0.0673 



77 



0.4974 
-0. 5061 
0.0673 



* 78 

0.6694 
.0.40*2 
0.0T37 



79 

0.8990 
0,4073 
0. 1701 



80 



1.2773 0.7220 
1.3546 2.381 1 
0.1308 0.1367 



82 * 

0.6362 
-1. 3637 
#.0673 



* 3 . 

0.3937 
-U9?J19 
0.0673 



84 

/ 

1»?484 
-0*74'81 
0,0933 



> 3. 




0.32 




0. 16 




0.59 


¥ 


0.?9 




.,,0.0? 




1.52 


¥ 


0.25 




8. 89 


¥ 


0.31 




0.18 




0.04 




o.tu 


2.* 




2.05 




2.6i 




0.84 




0.21 




. 0-*2. 


¥ 


0.96 




103 




C21 


¥ 


0.65 




1.05 


«• 


0^09 




0.02 


' 7.4 


¥ 


0.48 




0.43 




c.6> 




0.65 




0. 18 




1.23 




2.67 


¥ 


0.21 


¥ 


0.84 


¥ 


1.32 




t« 13, 




**23 


-2.0 




5.52 




2.40 




1.25 




0.04 


¥ 


0.21 




1.85 


¥ 


0.05 




0.03 




2.74 


¥ 


2.66 




0.05 


¥ 


0.5t 


1.6 








1.78 


. ¥ 


0.23 




0.23 


¥ 


3.Q2 




»0."2 




0.26 


¥ 


a. oo 


¥ 


0.00 


¥ 


1.12 




2.98 




0.74 


1.2 " 




0. 74 




0.74 


' ¥ 


0.01 


¥ 


2.Q$ 


¥ 


0.29 


¥ 


0.00 


¥ 


0.25 


¥ 


0.09 




0.26 


¥ 


2.19 




2.56 


¥ 


0.16 




«• 


4.50 




0.04 




0.91 


¥ 


2.75 


¥ 


1.04 


¥ 


0.09 




0.85 


¥ 


9.9% 


¥ 


1.9/ 


¥ 


3.8* 




2.91 




0.02 






«J • 92 




,0.35 




3.66 




0.17 




1.07 


/- 


0.34 


¥ 


1.55 




0. 39 




O.Ot) 




0.4-8 


«• 


3*92 


¥ 


0.63 






3.*1 


«• 


0.59 




0.8S 


¥ 


0.01 




*.84 




Q.'U 


¥ 


0.30 




0.63 


¥ 


0.03 




1.89 


. «■ 


V.67* 




4.28 


"•4 ' 




0.87 




1 .86 




0.43 




4.8,3 




i .00 


¥ 


0.1'7 




0.00 




3.88 




0.41 




1.93 


«■ 


3.4i; 


¥ 


0.49 


-• * 




1/31 




t.55 




0.56 




-5.94 




1 .09 




0.10 




0.97 


¥ 


0.58 




3.59 




0.81 




0.75;* 


¥ 


0,07 


-1.2 




6.19 




0.31 




1.95 




0.00 




0.22 


¥ 


0.04 




3.06 


¥ 


1.53 


> 


1.75 




0.30 




0.29 , 




0.31 


-1.6 




1.70 


f 


0.23 




0.38 




t 3.72* 


♦ 


3.^6 


¥ 


0.00 


¥ 


0.28 


¥ 


4.03 




0.45 




0.12 


¥ 


0.21 * 




0.02 


-2.o 


♦ 


2.90 


«• 


Q.23 




0.41 


¥j 


r 4.64 




0.61 




0.07 


♦ ¥ 


1.84 


¥ 


4.90 


¥ 


1.74 




9.4Q 




1.50' 


¥ 


a. 12 


-2.4 


4- 


0..)! 


«• 


9. *8 




0*82 


¥ 


6.10 


¥ 


2.0? 


¥ 


0.70 


¥ 


2.98 


¥ 


2.21 




0j02 




4.65 




9.06 




0.41 


-2.8 




0.80 


♦ 


7.7* 




0.07 




0.05 




0.46 


¥ 


1.51 




0.06 * 


¥ 


0.74 


¥ 


1.12 




5.16 




0.12, 


¥ 


o;35 


<-3. 


4- 


1.51 




4. 76 


¥ 


0,0 7 


¥ 


f .10 


¥ 


0.31 


¥ 


1. 12 


¥ 


8.89 


¥ 


0.08 


¥ 


0.35 




3.85 




0.08 


¥ 


4.09 


TOTAt CH! 




34.25 




35.15 




13.04 




36.68 




19.79 




Q.80 




22.5/ 




29.31 




16. 23^ 




41.08 




30. £2 




17.46 



Item 



* Item Numbers 



> 
a* 

H» 
rt 

n 

rt 
ro 
op 
o 
n 

CO 



rameters 




95* 




86 




K7 




8* * 




8Q 

1 


90 * 




91 




9? 


93 


94 




95 




96. 






1.1055 




1.2479 




1.0439 




0.4713 




1.0227 


0.8396 




1. 1620 




0.6454 


0..6785 


1.1100 


0.2828 


* V3^8676 


n 




0.1603 




0.4799 




0.0478 




0.3102 




1.4264 


rt.1618 




0.7679 




0.1979 


0.5859 


0.4267 




1.7099 




0.6004 


c 




0.14,15 




0.75O3 




0.17 OQ 




0.0673 


0. 1228 


0.1228 




0.2593 




0.2053 


0. 1839 


0. 1690 


0.0673 , 




0. 1702 


> 3. 


¥ 


0.04 


♦ 


0.05 


«• 


0.08 




, 1.24 


¥ 


0.01 


♦ 0.05 


¥ 


0.02 * 




' > 
0.79 


♦ 1.34 


♦ 0.11 


* 


0.83 


¥ 


0.47 


* 2.8 


¥ 


0.1? 


«• 


O.20 


«• 


0.24 




0.84 


¥ 


0.02 


♦ 0. ii 


¥ 


0.07 




0. 11 


♦ 0.14 


♦ 6.36 


¥ 


0.06 




0.61 


2.4 








*3.?2 


¥ 


0.00 


¥ 


0.1? 


¥ 


1 0.10 


- 5.78 


¥ 


0.33 




0.5O 


- 0.8? 


- 3:28 




2.72 


¥ 


0.08 


2.3 


¥ 


1.14 




0.66, 




0.37 




3.^0 


¥ 


0.43 


- 9.08 




1.28 


¥ 


3.64 


- o.n 


- 0.73 




0.71 




3.26 


1.6 




0.34 


«• 


0.04 




2.50 




5.34 




0.34 


- 10.89 


¥ 


0.64 




0. 1* 


. ♦ 0.13 


♦ 0.04 


¥ 


0.72 


¥ 


0.76 


1.2 


¥ 


0.>>6 




0.6 8 


¥ 


0 .00 




4.50 




0.57 


-f 6.78 


¥ 


4.14 




0. 10 


- 0.03 


♦ 0.65 


¥ 


1.75 




1.92 




¥ 


0.05 


♦ 


0.75 


\ ¥ 


1.44 




0.00 




3.32 


-J 10.42 




0.37 




0.00 


- 0.15 


- 0.00 


¥ 


0.14 


¥ 


1.27 


.4 


¥ 


0.91 


¥ 


0.22 




0.03 


¥ 


6.57 


¥ 


2.21 


& 0.06 




0.02 




0.08 


♦ 0.85 


¥ 1.01 




1.-75 


¥ 


6.24 


• 0 




1.49 


¥ 


1.12 


¥ 


0.43 


¥ 


10.94 


¥ 


1.32 


♦ 7.64 


¥ 


0.23 




0. 18 


.0.00 


- 1.43 


¥ 


0.02 




0i64 


* -.4 




# 1.28 




. 3.91 




2.93 


¥ 


-0.78 


¥ 


O.Ul 


♦ 10.28 




2.90 


¥ 


0.59 


- 2.47 


- 0.56 




"0.10. 




7.60 


-.8 


¥ 






0. 16 




1.07 




3.56 




0.47 


♦ 9.30 


¥ 


0.73 




0.40 


- coo 


♦ 0.1 1 




0.10 




0.72 


-1.2 


¥ 


o.do 


¥ 


0.9? 


¥ 


7.54 




1.05 




0.67 


- 3.16 


¥ 


1.25 


¥ 


0.00 


¥ 3.56 


¥ 0.68 


¥ 


0-.03- 


¥ 


4. 16 


-1.6 


¥ 


1 .7% 


¥ 


o.?f 


¥ 


1.59.., 




8.01 


¥ 


0.34 


- 24.21 




0.09 


¥ 


0.72 


♦ 1.31 


*■ 2.63 


¥ 


0.16 


¥ 


2.93 


* -2.0 




0.40 


¥ 


10.14 


¥ 


1*12 




0.10 




0.02 


- 4. 96 


¥ 


0.85* 


¥ 


0.75 


- 0.82 


¥ 0.08 


¥ 


0.80 


¥ 


2.59 


-2.4 


¥ 


9.35 


¥ 


2.91 


¥ 


■'*'d*.oi 




0.60* 




0.09 


« — 4.07 




0.01 


¥ 


0.31 


♦ 0.12 


♦t 0.27 


¥ 


0.00 




0.18 


-2.3 




0.00 


¥ 


1.66 




0.11 




0.53 


¥ 


1.63 


*- 0.39 


¥ 


6.69 


"0 


1.77 


¥ 0.11 


- 0.58 


¥ 


0.26 


¥ 


0.77 


<-3. 




2.03, 


¥ 


0.85 


¥ 


0.34 


¥ 


2.10. 


¥ 


4.59 


♦ 0.*6 




0.05 


¥ 


2.37 


- 0.91 


- 0.09 




1 .46 




0.25 


TOTAL CHI 




24.39 




27.21 




19.79 




50.17 




14.07 


107.04 




19.67 




11.95 


12.91 


12.63 




11.12 




34.45 




• Table 7 (cont'd) 



. Item 
Parameters 



Item Numbers 



<58 



tot 



» 

(0 
! 0Q 
*»• O 

rl 

(0 



4 


0.9017 




o. 7or* 


0.*323 




U2*«4 


, 0.4779 




0./167 




1.0162 




1. 1351 




1.942? 




2/1850 




o. ioo* 




0. 1001 


0.0111 


0. 1134 


0.0673 


> 3* 




0.51 




7.32 




2.61 




4.35 


*• 


0.07 




*■ 


108 




0.O3 




0.23 




0.09 




0.21 






4.44 




0.12 




£.78 




0.2* 


♦ 


0.7? 






0.59 


t 


3.0? * 




ft. 76 




2.91 




-4.02 




♦ 


1.32 




0.2* 




1.48 


♦ 


0.00 


*■ 


0.27 


1.2 




0.00 


*■ 


v 0.49 




0.71 




0.10 




0.00 


• a 




0.05 


*■ 


3. V? 




0.47 


♦ 


0.?2 




1.12 


.4 




0.09 




a. 34 


♦ 


0.0? 




1.01 




6.32 


• 0 


♦ 


0.05 




U33 




- 0.17 




?.I? 


♦ 


0.35 




*■ 


0.1/1 


♦ 


2.51 


♦ 


' 1.40 




0.02 


♦ 


'.17 






0.70 


*• 


C.43 




3.28 


♦ 


2.*5 


r 

*■ 


4.14 


-1.2 




0.79 




C22 




0.53 v 


♦ 


6.66 


♦ 


0.15 




*■ 


4.84 




0.27 




0.01 


♦ 


0.Q5 




3.82 


t -?.o 




0.15 




3.54 




0.17 




0.88 




0.19 


-2.4 


*■ 


1*13 




2.35 




} 0.69 


*■ 


0.92 




0*?6 


• -2.8 




0.02 




0.01 




0.30 




1.09 




0.28 


<-3. . 


*■ 


0.61 




0.01 


*> 


1.57 




0.07 




1.70 


TOTAL CHI 




16.6* 




*?4.57 




23.18 




23,67 




76.79 



102 

0.9738 
-0.9235 i 
0.06 73 

♦ .0.01 

- 30.75 

♦ 0. 15 

- 3.09 

- 0.38 

♦ 0.00 

♦ 0*05 

- ,1.11 

- #.0 3 

♦ i.9? 

♦ 0.03 

♦ 0-91 

- 0.81 

- 2,36 

♦ 0.34 

- 2.22^ 

- -0.69 



103 



104 



105 



106 



108 



94 



0.5100- 


0.8343 




1.3571 


0.6919 




1.1251 


0.4250 


-0.8485 


- 


0.3070 




1.7021 




2.3840 




1.2667 


- 


0.8533 


0.0673 


0.0026 




0.0908 


0.029? 


0.1283 


0.0673 


♦ 0. 18 


♦ 


0.05 


*■ 


0.21 


*■ 


0.64 




„ 0.16 




'1.24 


♦ 0.53 




0. 19 


*■ 


1.15 




. 6.04 




0.?2 




0.00 


- 0.03 




0.05 




1.03 


*■ 


0. 17 


♦ 


0.03 




0.32 


♦ 0.05 




0.38 




0.00 




0.27 




0.85 




%,60 


- 2.3Q 




1.25 


♦ 


0.00 




, 0.01 




0.51 , 




0.0Q 


- 1.43 




0.03 


♦ 


0..86 




0.01 


*■ 


0.80 


' *■ 


0.25 


♦ 0.38 


♦ 


1.08 




0.01 


♦ 


0.33 




0.62 


*■ 


0.62 


♦ 2.06 




4.07 




1.99 


♦ 


0.04 




n.34 




0.67 


o.oo 


♦ 


0.52 


*■ 


0.31 


r _ 


1.17 


*■ 


1.81 


*■ 


0.01 


- 0.75 


♦ 


1.89 




0.09 


*■ 


0.75 




0.30 


*■ 


1.19 


- 0.87 




0.03 


♦ 


3.28 


♦ 


0.12 




3.31 


*■ 


4.14 


♦ 5.40 


♦ 


0.2? 




0.46 




1.03 1 


*■ 


5.97 




0.34 


* 0.2^ 




5.75 




1.27 


♦ 


0.49 




, 0.34 




4.33 


- 0.56 




0.36 




0.<00 




0.42 


♦ 


0.07 




3.13 


- 5. 15 


♦ 


2.27 




2.2? 




6.04 


*■ 


7.38 




0.15 


r 0. 10 




0.26 


♦ 


1.95 


♦ 


" 0.51 


*■ 


0.92 


♦ 


0*59 


- 0.00 


♦ 


0.28 


♦ 


5.81 




0.19 




0.51 




Li5 


70.12 




18.69 




20.6*/ 




6.22 




24.13 




26^4 



• Item. — — 
Parameters 109 



rr 

a 
oj 
rt 
o 

OQ 

o 

H 
H- 
O 
CO 



Item Numbers 



no* 



111 



112 



113 



114* 



115 



116 



117 



A 


1 


.0282 


0.5737 


0.7729 




U5804 




1 .1986 




1.7500 




1.75 00 


0.80?1 




1.1056 


'ft 


0 


.1542 




0.4150 


0.9318 




1.4062 




1.5804 


"201 76 




2.3166. 


0*7871 




2.0503 


C 


0.221* 


0.067* 


0.172* 


0.1745 


0. ^695 


* 


0.1443 


0.0*93. 






0.1474 


> 3. 


♦ - 


0*03 


♦ 


0.20 




1.70 


♦ 


0.05 


♦ 


0.22 




0.75* 




^0.18 




2.64 




0.22 


2.3 


♦ 


0.12 




0.62 


♦ 


2.3% 


♦ 


0.32 




4.19 




8.67 




0.?9 




1.78 


♦ 


2.71 


2.4 


♦ 


0.6* 


♦ 


0.64 




O^U 




0.35 




4.87 


♦ 


2.53 




' o.no 




0.06 




0.93 


2.0 




0.88 




•*.58 




0.47 




i»49. 


♦ 


1.21 


♦ 


3.84 




0. 1? 




0.01 




0.00 


1.6 




4.9* 


♦ 


4.13 




0.05 




0.00 


♦ 


0.18 




3.08 , 




0.09 


¥ 


0.46 




0.00 


1.2 




0.23 


♦ 


2.24 




3.53 




1.65 




0.79- 


♦ 


0.01 




0.0/ 




3.07 




0.26 




♦ 


CO* 




0.29 




0.09 




0.52 




0.15 




2.*r 




1.67 




0. 14 




0^.07 


.4 


♦ 


5.96 




0.17 


♦ 


0.28 


♦ 


0.14 




0.36 




5r27 




0.51 


♦ 


0.39 


♦ 


0.59 


* -o 


♦ 


COO 




8.57 


♦ 


0.36 




0.29 




0.28 




10.53 




0.01 




0. 18 




1.03 


-.4 




5.54 




1.21 


♦ 


0.79 




1.35 




0.01 




' 0.91 




1.76 




1.36 




0.50 


-.8 




0.06 




l.<U 


♦ 


0.97 




0.42 


♦ 


1.17 


♦ 


0.42 




0.26 




0.43 


♦ 


0.23 


-1.7 


♦ 


1.20 


♦ 


5.93 




1.05 


♦ 


3.81 




0*>2 


♦ 


16.45 


♦ 


0.18 


♦ 


0.17 


> 


0.00 


-1.6 


♦ 


0.11 


♦ 


1.28 


♦ 


1.11 




0.02 


*• 


0."*2 




28. *2" 


♦ 


l.?3 




1.05 




0.01 


-2.0 


♦ 


5.23 


*■ 


9.40 




0.89 


♦ 


1.23 


♦ 


15^5 


♦ 


5.3* 


♦ 


6.02 




0.09 


♦ 


2.10 


~-2.4~ 




"0.06 


♦ 


0.78 




3.67 


♦ 


~ 4.?9 


♦ 


0.03 


♦ 


1.20 


♦ 


T.78 




6.40 


*■ 


4.09 


.' -2.8 




1.17 


*■ 


3.00 




0.** 


♦ 


1.00 




0-.12 


♦ 


2.08 


♦ 


3.13 


♦ 

* 


3.70 




0.56 


<-3. 




2.69 




1.44 




I.l3 k 


♦ 


2.13 




0.17 




0.24 




0. 19 




0.00 


♦ 


0.18 


TOTAL CHI 




23.96 




44.37 




19.39 




19.06 




30.23 




92.99 




17.51 




12.44 




13.47 



11* 

0.6412 
2.6 188 
0. 1359 

- 1.20 

♦ I.J9 

- 0.88 
¥ 2.96 

- 1.73 
» 0.08 

- 0.02 
» 0.62 

♦ 0.01 

- 2#10 

♦ 0.15 

- 0.38 

♦ 0.27 

► 7/30 

o;o9 

v 0.36 

> ^0.01 

19.34 



119 



170 



1.3*72 1.3113 
2.0110 1*2148 
0.0865 0.0913 



0.41 
5.00 
0.18 
0.73 
0.?7 
0.59 
4.3d 
0.15 
0.50 
' 0.?2 
0.05 
0.20 
0.37 
0.C0 
6.44 
0.07 
0.00 

19.55- 



0.07 
0.24 
0.01 
0.06 
0.25 
0.?0 
0.83 
1.06 
0.59 
1.74 
0.05 
0.84 
0.95 
1.45 
0.72 
0.03 
1.42 

10.51 



9 

ERLC 



4'J t 



Table 7 (cont'd) 



> Item. 
Parameters 



Item_Numbers 



H* 
rr 

O 
(0 
rr 
(D 
OQ 
O 
i-t 
H* 
fl> 
GO 



A 

n 

c - . 

> 3. 
2.8 
2.* 

1.6 
1.2 
' .8 
.* 
.0 

..-•4 
' -.8 
-1.2 
-1.6 ' 
-2.0 
-2.* 
-2>.8 
<-3. 



TOTAL CHI 






























121 




122 




123* 




124 




1?5 




126 




127 


).641> 




0.97*0 




0.5208 




1.3795 




0.9550 




1.2157 


0.9842 


-7.2698 




0.5415 




0.0772 




0.0218 




1.8790 




1.2520 




1.8806 


' T.')673 




0.33*8 


*0.0673 


0.1691 




0 f 2l01 , 
i 


0. 1115 


0. 1262 


0.03 




0*11. 




0.65 


¥ 


0.01 


¥ 


1*08 


♦ 


0.1S 


•> 


0.00 


♦ 0.06 




1.70' 


¥ 


1.23 




0.02 




• '1.10 


♦ 


0.65 




0.00 


♦ 0.22 




0.05 


¥ 


2.08 




0.12 


* 


o'.oi 


* 


0.42 ' 




0.07* 


- 0.03 


¥ 


4.13 




0.25 




0.74 




0.80 




0.37 




0.15 


- *jQ50 




C.08 




1.73 


» 


o.a3 




3.21 




0.11 


•> 


0.09 


♦ >,*)3 




2.7* 




0.33 




p.OJ 




0.22 




0.2.1 




0.34 


- 0.17 




0.46* 




0.C3 


«• 


0.73 




0.37 


> 


0.28 


¥ 


1.59 


- 1.50 




C.6* 




0.02 




0.72' 




0.26 




0i37 




0.04 


- 0.43 




0.98 




2.99 


«• 


0.00 




5*. 19 




0.63 




0.30 


K/ 5.49 




1.10 




0.37 




0.01 


¥ 


0.33 


* 


1.71 




I. 11 


< 0.37 




0.92 




0.02 


«• 


1.09 




1.25 


•> 


0.*7 


¥ 


1.10 


♦ *1.61 




5.84 




0:08 




0.05 




0.46 


* 


0.00 




lV»9 


- 0.41 


•> 


0.07 




0.28 




0.05 




4.13 




0.83 


¥ 


2.87 


- 5.66 




0.03 


¥ 


8.03 




2.1? 




0.11 




1.82 


¥ 


0.26 


- 0.46 


♦ 


0.28 




.5.81 


¥ 


0.C3 




o.os 




0.29 


+ 


1 2.6*8 


- 4. 76 




05 14* 




10.63 


* 


0.62 




1.66 




1.06 


f 


• 0.13 


- 1.41 




0.00 




U.04 




3.35 




o 2.6? 




O.08_ 


f 


3.80 


22.35 




19.25 




34.57 




9.20 




24.86 




10.11 




16.53 



128 £ 




129 














1 . 7500 








1.39*6 


2.3687 




1.4698 




0.501? 


0.0716 




o.i??a 




0.0809 
• 


♦ 0.89 




0.00 


¥ 


0.00 


♦ 2.20 




0.01 


¥ 


0.01 


+ 0.01 


¥ 


' 0.06 


¥ 


0.04 


- 0.03 


¥ 


0.28 




2.30 


- 0.00 


¥ 


0.8* 


¥ 


0,06 


- 2. 18 


¥ 


0.17 




3.54 


- 6.45 


¥ 


0.00 




1.35 


- 6.77 


¥ 


0.79 


¥ 


' o.po 


*, 0.07 




1.75 


¥ 


0.07 


- 0.30 




1.47 


¥ 


9.17 


♦ 11*82 


¥ 


a.09 




1.0* 


♦ 1.37 




2.81 




7.04 


♦ 0.23 


¥ 


o. id 




2.98 


♦ 1.29 




0.72 


¥ 


0.66 


♦ 2.81 




0.51 




0.12 


♦ 4.83 




0.,36 




8.65 


♦ -0.01 




0.72 


¥ 


2.81 


41.27 




10.67 




39.83 



1-31 - 132 



0.6162 
-0*9,196 
0. 1228 

♦ 0.15 

♦ 0.30 

- 0.98 

♦ 2*1? 

- 0.01 

- 0.57 

♦ 1.09 

- 0.03 

♦ 0.28 

- 2. 19 

- 0.?8 

- 0.02* 

♦ 6.** 

- 0.16 

- 1.32 

♦ 0.02' 

♦ 0.07 

16.04 



0.9537 
0.4231 
0.1307 

» 0.12 

- 8.91 

- 0*3f5 

- 2.62 

♦ 0.01 

♦ 0.08 

♦ 0.47 
•> 0.58 

♦ 0.01 

- 0.18 

- 4.01 

- 0.14 
► 2.23 

♦ 0.23 

♦ 0.23 

♦ 3.92 
r 0.95 

25*03 



Item Numbers 




Item 












135 




1*6* 






















rameters 




133 




134 








137 




138 




139 








1*1 


































A -***^0.63*0 


019182 




1.192*4 




U0O*3 




1.0868 




0.7648 




0.6656 




1.2325 




1.0*0* 


8 


1X94*2 


0.66A'0 




1.8029 




t.5843 




1.6?4* 




2.2051 




1.89 32 




1.7917 




2.5579 


c 




0.1 1<>7 




0.2861 




0.2393 




0. 10*6 




1.1535 




0.190? 




0.1259 




0.O326 




0. 0* Q Q 


> 3w 




0.26 


¥ 


0.16 


¥ 


0.62 


¥ 


0.52 


¥ 


0'.54 


¥ 


0.75 


¥ 


0.*6 




0.52 




3.03 


2.8 




0.O1 




0.^4 




0.01 




0.1*8 




7.49 


¥ 


1.7? 


¥ 


1.22 


¥ 


0.*9 




0.65 


(r 2.4 


¥ 


0.08 




0.04 








0.55 


¥ 


" y.64 




0.46 




l.*7 


¥ 


1.72 




0.00 


2.0 




3.01 




0.01 


¥ 


O.pl 


¥ 


0.00 




0.19 


¥ 


O.O* 




0.03 




0.6? 


¥ 


1.64- 


1.6 




0.47 


¥ 


0.12 




0.27 




<V85 




0.02 




0.77 




Jo 0.02 




0.09 


¥ 


1.66 


1.2 


¥ 


0.14 


¥- 


o.&n 




1.36 




0.67 


¥ 


0.01 




0.00 


¥ 


0.22 




0.12 


¥ 


0. 15 


.8 


¥ 


3.63 


¥ 


0.00 




0.93 


¥ 


1.51 




0.18 




0.05 




9,36 








f>.97 


• 4 




2.66 




0.02 


¥ 


0.78 




0.*5 


¥ 


0.11 


¥ 


0.00 


¥ 


0.4* 




0.06 




fl.23 


' .0 




0.>5 




1.7* 


¥ 


0.*7 




0.16 




0.14 


¥ 


0.50 




0.0? 


¥ 


*.71 


¥ 


0.00 


-.4 




0.26 


¥ 


0.35 




2:26 




1.72 


¥ 


1.99 


¥ 


2.8* 


¥ 


. 0.25 




\).?5 




C.*3 


!-.8 


¥ 


0.39 


¥ 


o.uf 




0.09 




0.29 




1.08 




1.27 




0,07 


¥ 


2.17 


¥ 


0.10 


-1.2 


¥ 


0.17 


¥ 


3.67 


¥ 


0.47 


¥ 


0.01 




0.13 




0.07 




3.12 




2.*? 


¥ 


5.25 


•1.6 




0.22 


¥ 


0.09 


¥ 


3.57 


¥ 


0.18 


¥ 


1.42 


¥ 


0.40 


¥ 


0.30 




2.. 78 


¥ 


7.55 


-2.0 




o.co 




0.40 


¥ 


0.'*f 


¥ 


1.9*1 


¥ 


0.06 




1.13 


¥ 


0*2? 




0.05 


¥ 


0.13 


-2.4 




0.O5 




0.62 


¥ 


0.01 


¥ 


'2.40 


¥ 


0.85^*- 


- 0.11 


¥ 


O.07 


¥ 


6.01 




0.58 


-*.8 


¥ 


1.14 


¥ 


1.73 




1.37 


' ¥ 


23.05 


¥ 


0.57 


¥ 


0.17 


¥ 


1.01 


¥ 


2.16 


¥ 


0.95 


<-3. 


¥ 
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31.86 



• Items appear in , the following order: * • ^ 

PSAT/NMSQT Form 1, items 1-50; ? 
PSAT/NMSQT Form 1\ items 51-100; 
SAT Firist Old Form, items 101-120; 

SAT Second Old Form, items 121-141. - 

2 Asterisks next to item numbers indicate items with questionable fit td the t|uree- 
parameter logistic model. 
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Figure 10: PSAT/NMSQT Form ,1 Item Ability Regression Plots 
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Figure 11: PSAT/NMSQT Form 1 Itfem Ability Regression Plots 
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Figure 13; PSAT/NMSQT Form 1 JLtem_ Ability Regression 'Plots 




" Figure 17: PSAT/NASQT Form 2 Item Ability '.Regression Plots 




Figure 26»: SAT First Old Form Item Ability Regression Plots 
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v Figure 27 i SAT First Old Form Item Ability Regression Plots* 
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Figure 28: SAT Second Old Form Item Ability Regression Plots 




Figure 29j SAT Second Old Form Item Ability Regression Plots 




( It should be noted, when inspecting the information in Table 7, that 

the interval contributions have signs attached to them. One of the weaknesses 
of tbe use of chi-square like statistics to' assess model-data fit is that the 

statistics do not take dnto account whether or not the deviations of 

>~ " ■» % 

observed values from expected values are in a single direction. This is 
particularly impprfcant as an erratic pattern of pluses *and minuses may be 
indicative of random fluctuations in the data, wher&as a number of pluses^ ( 
(or minuses) ocurring in a row indicate that the empirical degression is 
cdnsistently above (or below) the estimated regression for specific ranges 
of the ability continuum.'* * 
For e-ach item, the information obtained from the Q f - statistic was used 

* J. * 

in conjunction with the item ability regression pltft (Figures 6^29) obtained 
• '* " < • • * 

for that item to determine goodness of fit. , The process consisted of first, 
inspecting the item ability regression plot, to form an opinion as to how 
well the item fit the data. If the mid-points of a^series of boxes, repre- 




sensing the empirical regression, fell outside the + PQ/N.. interval 
centered on the response function, the value of Q^' (particularly the contri- 
butions of the ability .level., intervals and , the sign of these contributions) 
was inspected .to form & judgement, as to whether the item was truly poorly ^ 
fitting. Secondly, the process was reversed and the' Q^ f statistic for a 
particular item was inspected and used as a flagging mechanism to determine 
if the item ability regression 'plot warcantei close inspection. If the •• 
value of the statistic was large in relationship to values obtained for other 
items, the contributions of the ability level intervals was studied^ k£ 
opinion was formed regarding the goodness of fit of the item and an attempt 
was made to verify this opinion by examination of the item ability regression 
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plot* Using t^^fcback and forth procedurejdfche authors were able to isolate 



24 Items for which, the gpodness of fit to tte three parameter model was\ 
judged questionable. These items are indicated in Table 7 by an asterisk 
placed next to the item numbey. Several of these items will be discufsed 
in the following paragraphs. ' ' t - > 

Examination of the data contained in Table 7 indicates that the value 



or 



Q 1 for item I (PSAt/NM$QT* Form 1) is '65.&l\ t Although selec^d ability' * * 




level intervals (0 = 1.6, 0 » «.8) are contributing heavily to the Value of 
this statistic, other intervals are also making contributions .of considerable 
size. The f air ly^consis tent -^t£em of'pluses and , minuses is indicative 
of the/rafcges on the ability- continuum |or which the item fits poorly. The 
value of -the c parameter for this item was set to .067. Inspection of the 
item ability 'regfce.ssiort plot shown in Figure 6'indicates that had tiie c 
parameter been Allowed to assume a lower value, fit would* have been, improved 
for lower ability levels, fhis would, not, however,. hav$ improved fit at 
•the upperiend of *t'he" ability continuum.. n * 

9 - The item<ability regression plot for item 5 (PSAT/NMSQT Form 1), given 



jraja, indicates that the empirical regression falls above the esti- 

\ ' * '.• * • / "\ \' ' ' * '' N. • 

'mat^f regression for ability levels* below r 2. On the other hand, for ability 

* . / ; • 

JLeyels just'belp^ 0 = 0 and ability levels above 0 =* +2, the empirical 
regression falls below the estimated regression. The valued of the 
statistic for this item (giv^n- in Tabled) is 32.71. N An examination of the 
contributions for the' ability levei intervals verifies the- pattern fchat is 
displayed by .the item ability regressiqnSplo.t, iae., the observed data' does 
not fit a monitonically indreasijig function very well. Onfe might question » 
why ltem5 was selected and item 4 .(BSAT/NMSQT Fo;rm 1) was not, given that ^ . 



• > 



the value of Q. J is slightly iarger for item 4 thkn for item 5. There are 

two reasons for this. First, it would appear, from the item ability regression 

plots, that item 4 fits the model better than item 5 (fewer boxes fell out 



of the interval defined by + 2/PQ/N,). Secondly, - from. examination of the 

2 * 

contributions of the ability level intervals to the overall value of Q^ f 
for item 4,*it is noteworthy that only one category (8 = 2.8) is contributing 
4 qv^r half of the value of the statistic. % ^ 

* * The item ability regression plot* far item 18 <PSAT/NMSQT Form 1) shown 
in Figure 8 indicates some poor fit at the extremes of the ability-continuum, 
however, from the appearance of the plot it would not be expected that the 
value of 0' would be as lafrge as 223.64 (the lafegestLvalue obtained for any 
item in this study). Examination of the individual ability level interval 
contributions to the overall chi-square valu& inditates that the interval 
containing responses of examinees with 8 greater than +3 has a value of 
-145.15. Of the twelve examinees in this interval, only one answered the 
item incorrectly inflating tfre contribution of this interval to the overall 
statistic .unreasonably, The item^ was- judged to be poorly fitting because 
of the fairly large contributions of the remaining intervals and^ the. con- 
sistent pattern of the plus and minus signs for these intervals. 
/ The final example of a poorly 'fitting item that will be discusse.d is . 
item 44 (PSAT/NMSQT Form 1). The item ability regression plot for this 
item given in Figure 13 clearly shows the non-monotonicity of the data. 
The fact that this is a poorly fitting item may be verified by examination 
of the chi-square information given in Table 7. No single ability level 
interval seems to„be contributing unreasonably to the overall value of 69.03. 

■ ": t- ' 

Also, the consistent pattern of pluses and minuses reflects the non- 
monotorticity observed in the item ability regression plot. 
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% In summary, a few general comments can be made,, First, the 'majority 
items judged to have questionable^fi£xto 'the three-parameter model were' 
PSAT/NMSQT items. It is likely that this is an artifact of the particular 
' forms chosen. Were the experiment to be repeated with four different forms, 

the greatest number of poorly fitting items might have been found in the 
, SAT forms. \ ' 

Secondly, it appears as though the majority 1 of PSAT/NMSQT and SAT 
items judged as poorly fitting basically exhibited poor fit for the extremes 
of the ability continuum, patticularly f or lower ability levels v This is . 
^fmost probably related to the problem of obtaining an adequate estimate of 
the psuedo-guessing parameter " of the" tfiree-parameter logistic model.' 



Discussion 



The equating 'models examined in this- study are based on a number- of 
assumptions, most of which are violated to some extent. One of the assump- 
tibnS underlying all of the equating models is that the two £ests to be % 
■ eqiiated are unidimensional (Morris, in press). Because the IRT equating 
model 4ssumes unidimensionality op. the item level whereas the linear and 
equ^percentilq models use% for this study .only assume unidimensionality 
of test scores, one might expect violations of this assumption "to have a . 
more serious effect on the IRT equating results. However, unidimensionality 
is a necessary condition for the establishment of a single common metric 
regardless of the equating model. Thus, it is difficult to say "what the 
implications of Violation of this assumption are for any of the equatings. 
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The equatings ,are certainly affected in a differential manner j>y 
availability of data< Insufficient data at low ability levels generally m 
leads to problems in estimating the psuedo-guessing parameter for the three- 
parameter logistic model. It is 'als^dif f icult to estimate discrimination:' 
► parameters apd difficulty parameters for very easy or very difficult items 
if adequate data does not exist. Equi^ercentile equating methods are 
particularly sensitive to lac]|t of ■ data at the extremes of the ability 

M r 

continuum. ~~ Scarcity of data at these extremes can lead to serious problems^, 

in establishing- an equitable relationship for high and low scores on the 
• % , * 

two test forms* The linear equating method, which is "based on estimated r 

means and standard deviations, can also be affected by the influence of 

outlying values on these estimates. This was probably not a problem for 

i 

the present study, gj.ven,that sample sizes were quite large and should have 

V * * • ... 

J^een sufficient to produce stable estimates. 

\ • 

Differences in test reliability must also be considered, tord (1980, 
Chapter 13) states that in order to accurately equate twb tests, i.e. jproduce 
'"scores on two tests such that it is a matter of indifference to Examinees 
which test they take, the tests must be strietly 'parallel and perfectly 
reliable. It is difficult to predict how the" various equating methods are 

• , x ~ i 

affected "by differences in test reliability (certainly a problem for this 
study, given the differences in test length). "However, the methods based 
on true-;score estimates (IRT and^Xevine Unequally Reliable) should be 
„ least affected by the prdblem. 

iW lack of parallelism between the, t;ests to-be equated has paftjLcular 

JLmpMcations for the linear method which requires, in order to adequately 

v 7 * 

describe the relationship between scores on u two forms of a test, that the 

distributions of these scores differ «only in their means and standard 
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deviarions. Lack of parallelism between two tests generally results in a % 

•curvilinear relationship between raw score^-necessitating an equating method 

- ' ' ' * S 

such as IRT or equipercentile to produce accurate results. 

One mu3t also consider the problems posed by the differences in the t 

level of" ability of the PSAT/NMSQT and SAT groups. The purpose of anchor '\ " 

test, designs is to provide a mechanism to adjust *f or these differences. 

/ Marco, Petersen and Stewart (1979, in press) report that 'as group differences 
become more pronounced, the quality of the equating suffers. The results s 
of their study indicate that IRT methods, based on item parameters that are 

# group invariant, may produce better results in this situation. 

In the absence of an adequate criterion upon which to judge the equatings, 

i 

one must rely heavily upon the arguments described above to draw any conclu- 
sions" regarding the results of the present study. It is- apparent, from 

& • 

examination of the equating results,, that the relationship between t'he raw 

* • - ' * 

scores on the PSAT/NMSQT and SAT is curvilinear, particularly 'for the upper 

and lower ends and somewhat for the middle^of the score range ,\ Therefore, 

* the ^Tucker and Levine Unequally Reliable linear methods are probably inadequate 

for describing the raw score relationship between the PSAt/NMSQT and' the 

. • . . * t • 

*SAT. 'Because the equipercentile method is so sensitive to scarcity *of data", 
at 'the extremes of the score range* the IRT method woul<l appear to describe 
the. curvilinear relationship in a more appropriat^manner . This is most, 
probably true for the upper end of the score rarfge. A decision aa to wttich 
equating method (IRT or equipercentile) 1 is most, effective' at 'the lower end 
of Che sTSfore range is. difficult to make. Both methods are. affected by 
scarcity of data; accuracy of the IRT method depends on how adequately the 
c parameters yere estimated. ' As indicated by the gobdness of fit assessment, 



problems with the estimation of this parameter contributed, in some cases, 

to the questionable fit of items. 

It is probably^ safe to concludes from the goodness of fit assessment 

that, in spite of some- problems* related to the estimation of j:he psuedo-guessing 

parameter, the data fit the' three-parameter model fairly well. Certainly, 

this aspect of the study suffered from some methodological problems. 'The ♦ 

manner in which the ability jpontinuum was divided into intervals for the 

chi-square statistic was a particular problem. Some method of accounting 

> **** 

for within interval ylriance, such as subtracting the variance of the predicted 
proportion passing the item for a specific interval from the denominator* 
of the chi-rsquare statistic (Wright and Mead, 1977), should probably have 
been considered. Ultimately >f thfe decision as to whether~or not an item 
fit the three-parameter model .was a judgmental one. This could noti be 
avoided, given the present * state of the art. ^ 

Several of the problems encountered in the study are currently beiflg 
investigated at Educational Testing Service* The LOGIST program has recently 
been revised to improve the estimates obtained for the psuedo-guessing 
parameter. Although it was not possible to repeat the entire study (speci- 
fically^the equatihgs), item parameter .estimates and item ability regression 

plots were obtained for 'the same set of data using the revised program and 

> 

compared to those used for the present study. Substantial improvement was - 
found in the estimates obtained for the £ parameters for selected items. 

It &£ planned- to revise the goodness of fit statistic in the near future 
so that some of the problems encountered wi^h* its^p^lication in this study 
will be avoided. Hopefully systematic Xise of this statistic in conjunction 
with item ability regression plo^will provide information regarding model- 
data fit that can be studied on ta continuing basis; 
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lo summarize, the conclusions that can be drawn 'from the present $tudy 
regarding the appropriateness of the three-parameter model for a vertical 
equating situation and the goodness of fit of the data to this model are 
•limited* There are two reasons for the limitations: (1) the Jack of a 
criterion for judging the equatings; and (2) the subjective nature of the' 
goodness of fit assessment • Further research is certainly needed. An 
important direction for this research 1 * to take is the direct assessment of the 
affect of the number and degree ,of poorly fitting items- on IRT equating 
results. This assessment should take into account such variables as equating 
design, differences in groCip ability and lack of parallelism between the 
tests to be equated. 

More generally, the extent to which IRT methods are "being implemented 

in practical testing g>tj££tions has dramatically increased over the past 

several years. I,t is important' that^ researchers interested in the quality 

of the results of the many applications rigorously pursue the quest-ion of 

goodness of fit and the possible implications of lack of fit for the specific 

* * 

applications they are interested in. 
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